Code coverage analysis is an important ingredient of good software engineering and was amongst the first techniques invented for systematic software testing. Code coverage measures the degree to which the source code of a program has been tested. After a run of the program, a code coverage tool reports which parts of the code were exercised during the run. This information can be used for determining which portions of the program have not been run. If a product is released with some parts of the code that were never exercised during systems tests, it is highly likely that the program will fail due to software bugs that exist in those untested portions of the program. One of the goals of good software engineering practice is to achieve very high (if not 100%) code coverage during tests to minimize the risk of software failure in the field. It is important to note that achieving 100% code coverage is difficult due to diminishing returns, but the knowledge of code coverage is important to make the proper judgment of the coverage necessary before releasing a product.
Code coverage analysis is an important ingredient of good software engineering and was amongst the first techniques invented for systematic software testing. Code coverage measures the degree to which the source code of a program has been tested. After a run of the program, a code coverage tool reports which parts of the code were exercised during the run. This information can be used for determining which portions of the program have not been run. If a product is released with some parts of the code that were never exercised during systems tests, it is highly likely that the program will fail due to software bugs that exist in those untested portions of the program. One of the goals of good software engineering practice is to achieve very high (if not 100%) code coverage during tests to minimize the risk of software failure in the field. It is important to note that achieving 100% code coverage is difficult due to diminishing returns, but the knowledge of code coverage is important to make the proper judgment of the coverage necessary before releasing a product.
There are many approaches to code coverage measurement. One method is to tap the code segment addresses being requested by the processor. This typically requires hardware modifications but is the most efficient as it does not require changes to the applications. Since hardware changes are much more cumbersome, software methods are mostly favored.
Software methods of code coverage typically take the source code of a program and instrument the source code to introduce extra instructions which will record the execution of different parts of the program. This is typically done at the preprocessing stage before the code is converted into a machine-dependent executable. Hence, this method is highly portable and has been implemented by numerous software vendors like IBM Rational, GNU (gcov), LDRA, etc.
It has been noticed that some complex systems are being developed without leveraging any code coverage tools. One of the most important reasons is the overhead of code coverage tools on execution efficiency. The instrumentation introduced by code coverage tools can slow down the execution of complex software considerably leading to different behavior than what is expected without the instrumentation. This nullifies the relevance of the code coverage analysis. Even when the behavior is the same, the mere fact that the instrumented code takes a longer time to run, is seen as an extra expense (in terms of person and machine hours) that needs to be justified by the potential gains.
Other problems that are routinely faced include the memory, disk, and bandwidth requirements introduced by such instrumentation. In the case of small on-processor caches, the instrumentation may also cause cache pollution lowering the performance via increased cycles-per-instruction. Multi-threaded, multi-processor scenarios only compound the problems further.
It would be beneficial to have dynamic code coverage with lower overhead requirements.